Merge Times and Hitting Times of Time-inhomogeneous Markov Chains

The purpose of this thesis is to study the long term behavior of time-inhomogeneous Markov chains. We analyze under what conditions they converge, in what sense they converge and what the rate of convergence should be. A Markov chain is a random process with the memoryless property: the next state only depends on the current state, and not on the sequence of events that preceded it. Time-inhomogeneous Markov chains refer to chains with different transition probability matrices at each step. What makes them interesting is that they don’t necessarily have stationary distributions. Instead of comparing the chain to the stationary distribution, we look at the distance between two distributions started at different initial states. We refer to the time until the two distributions get sufficiently close as merge time. As a foundation for our simulations and proofs, we first show the convergence theorem for time-inhomogeneous Markov chains with a sufficient assumption. We then study the various ways of perturbing the random walk on the n-cycle, by forcing the random walker to move in a certain direction with higher probability if it is at a particular site. Changing perturbations at every step results in different one-step transition kernels throughout the chain, therefore making the chain timeinhomogeneous. We then compare the merge times, as a function of the number of states n, to those of the unperturbed time-homogeneous simple random walk on the n-cycle.